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ATTRITION REPORTING IN NAVY TECHNICAL TRAINING 



ABSTRACT 

This study examines academic review board (ARB) decision making 
in Navy technical training in relation to standardization in the 
system for reporting training attrition. The results of a survey 
show differences in ARB procedures across the schools sampled, and 
that there is fairly high agreement among ARB members about the 
value of the factors they consider in deciding to setback or 
attrite students in training. The study also includes a discussion 
of ARB member perceptions of the effects of the Navy's policy to 
keep training attrition low. The recommendations address the 
differences found, but also point out that the accuracy of ARB 
decisions is still unknown. A study of the costs and fleet 
performance of marginal students v.’ould provide a concrete basis for 
establishing attrition policy. 
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I. 



INTRODUCTION 



The loss of resources through attrition from Navy technical 
training programs is a concern, particularly at the initial skill 
training level in the "A" schools. Interventions designed to 
decrease attrition are driven by the attrition reporting system, 
which provides the data to show the specific causes that must be 
addressed. In this context, the Navy is dependent upon accuracy 
and standardization in attrition reporting. 

Academic Review Boards (ARBs) within each Navy school may 
recommend' attrition of students due to poor academic performance 
resulting from any of several causes. The purpose of this research 
is to investigate standardization in ARB decision making 
procedures . 



A. BACKGROUND 

Navy training commands provide specialized skill training for 
a wide variety of jobs for enlisted personnel. Training levels 
range from initial rate training at "A" schools to progression 
training at the more advanced "C" schools and highly technical "F" 
schools. For the most part, new recruits do not enter the Navy 
with the skills required to perform effectively on the job. 
Therefore, most graduates from Recruit Training Commands (RTCs) go 
directly to "A" schools. 

"A" school training is the most cost-effective means of 
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training recruits for most of their initial assignments in the 
fleet. [Ref. 1] The projected student input totals for "A" school 
attendance are 128,049 for FY91 and 126,603 for FY92. However, the 
projected graduation totals are 117,411 and 116,161, respectively, 
for the same time period. [Ref. 2] The difference of the two 
totals (over 10,000 students for each year) represents a 
significant loss of resources in the form of student drops from 
training . 

In relation to "A" schools, "C" school attrition is not a major 
concern. The current average of "C" school attrition is 3.0 
percent. There are a some significant differences between "A" 
school students and "C school students, which might explain the 
differences in average attrition rates. "A" school students 
usually come directly from RTCs, most are 18 to 20 years of age, 
they have not fully acclimated themselves to the military way of 
life, they may not possess the discipline to succeed in the 
military training environment, and as the training progresses they 
may question their desire to obtain that particular skill. 
Conversely, ”C" school students usually come from fleet units, they 
have already completed their initial skill training, and have added 
fleet experience to that knowledge base. These students have been 
closely evaluated by their superiors in the fleet and have been 
recommended for follow-on training. Consequently, when "C" school 
students arrive for training, most have the maturity, discipline, 
and motivation to complete their course 
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of instruction. 



Thus, the Navy's concern for attrition from specialized skill 
training is focused primarily on the "A" schools where student 
losses are the highest. Considerable attention has been paid to 
reducing "A" school attrition through research and development, and 
working groups have been formed to address the problem. 

Most of the research on "A" school attrition has looked at 
the effects that student characteristics such as mental ability or 
level of education have on academic and nonacademic attrition. 
[Ref. 3] Other studies have examined the impact of technical and 
non-technical courses on academic and nonacademic attrition 
patterns. [Refs. 4,5] 

In 1987, the Chief of Naval Education and Training (CNET) , 
established a "Model Schools Program." [Ref. 6] The intent of the 
program was to improve training by 

...bringing available resources into contact with a Navy 
school and collectively working together with the school 
management staff to identify problems that impede school 
success and develop solutions that can be implemented by the 

school staff. [Ref. 7] 

The EM-A school was designated as the first model school in which 
activities used to improve training in this environment could be 
transitioned to other technical training schools. Given the broad 
spectrum of improvements introduced to training in the model 
schools context, reduced attrition would not be an unexpected 
result for the schools that become part of the program. 
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Another CNET group, the Training Efficacy Quality Management 
Board, sponsors and distributes a lessons learned 

letter to its training commands. The letter is a compilation of 
the effective actions taken by the commands that have been 
successful in curbing their schools' attrition. The letter allows 
the schools to review the actions taken by other schools. [Ref. 
8 ] 

The purpose of the present research effort is to explore how 
decisions are made to drop students from training, and to determine 
if there is standardization in the decision making process. 
Accuracy and standardization in these decisions are essential in 
order to develop appropriate programs to reduce attrition. 

1 . "A" School Training 

Navy "A" school student enrollment is comprised primarily of 
students coming directly from the RTCs, although a small percentage 
comes from fleet units. Each student must meet some minimum 
entrance requirement to be admitted into the school — usually a 
minimum total Armed Forces Vocational Aptitude Battery (ASVAB) 
score or a minimum score on several of the ASVAB sub-tests. The 
newly reporting student participates in school indoctrination, 
which may last from several days to three weeks depending on the 
school and the student. Part of the indoctrination phase may 
require a battery of reading and arithmetic tests. The schools use 
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these tests to determine the student's knowledge of the basic 
skills, which are necessary to complete the training program. 
Students who do not pass these exams remain in the 
indoctrination phase and receive remediation on their skill 
deficiencies. Students who fail remediation do not continue with 
the training pipeline and eventually are reassigned to other duty. 
Students who continue with training are assigned to an academic 
class and an "A" School Military Training Company (ASMT) . The ASMT 
provides the students with a continuation of the general military 
training and physical fitness conditioning they received as 
recruits . 

The "A" school companies maintain a record on each student 
consisting of all the military training the student receives, any 
violations of school policy, military deficiencies, and any other 
personal information that may seem necessary. If a student commits 
an offense against school policy, he/she may go before a military 
review board (MRB) . The MRBs are used to correct the student's 
military deficiencies; however, if the problem persists, the board 
can recommend the student be dropped from training. Offenses of 
a more serious nature are handled at Officer in Charge Mast or 
Captain's Mast; both can result in dropping a student from 
training . 

The academic curriculum of an "A" school may be divided into 
phases. The school may have both military instructors, usually 
Second Class Petty Officers and above, and contracted civilians. 
The instructors teach just one segment of the 
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training phase, and a training phase may be made up of many 
training segments. Students are tested upon completion of each 
segment of training. Students who fail the test receive 
remediation including a reexamination. Students who pass 
remediation usually continue with their class, students who fail 
receive a mandatory Academic Review Board (ARB) . 

2 . Academic Review Boards 

"All Apprentice Training, Class 'AP, ' 'A, • and NEC awarding 

'C schools are required to convene ARBs for attrition and setback 
decisions." [Ref. 9:p. 2] The primary function of the ARB is to 
enhance student academic progress and to deter student failure. 
Along with that primary duty, the board must make an unbiased 
determination of whether the student has the motivation and ability 
to complete the training. An ARB is initiated by the student's 
Phase Chief at the request of the student's instructor for a given 
training segment. An ARB may be convened for any one of the 
following five reasons: (1) the student is recommended for 
acceleration through training; (2) the student fails to meet course 
learning objectives and after remedial study fails the retests; (3) 
the student continually fails course learning objectives, even 
though he/she passes all remedial exams; (4) the student's 
laboratory performance is consistently below standard; or (5) the 
student fails remediation for a learning objective following an ARB 
recommendation to continue with the 



6 



class. [Ref. 9] 

The ARB consists of four members chosen from the instructional 
staff, which includes officer and enlisted 
instructional/supervisory personnel, classroom and learning center 
instructors, and education/training specialists. 

Supervisory personnel who have command designated authority for 
approval/disapproval of ARB recommendations may not sit as members 
of the ARB. Those personnel responsible for completing student 
personnel and reclassification actions may not sit as ARB members. 
The board is chaired by the senior member and there are two other 
voting members as well as one non-voting recorder. After examining 
all the available relevant data, the board conducts an interview 
with the student. During the interview, the board attempts to gain 
information on any problems the student may be having that could 
cause the student's academic problems. Once the ARB is satisfied 
that they have reviewed all the available information, the board 
makes a recommendation on the student's future training status. 
If the board is able to determine the student's problem, it can 
initiate appropriate corrective actions. However, the board must 
make one of four recommendations: (1) continue the student with 
class, with or without remediation; (2) setback to the next class, 
with or without remediation; (3) accelerate to the next class; or 
(4) drop from training. 

If the board's recommendation is to drop the student from 
training, it must also make a further recommendation to 
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transfer the student directly to the fleet for general detail, 
reclassify the student to another rating, or separate the student 
from service. For the board to recommend that the student continue 
with the class or setback to another class, there must be clear 
evidence that the student has the ability and the motivation to 
complete the training. To recommend a 

student be dropped from training, the student must show an 
unwillingness or an inability to complete the training. 

The ARB's recommendation is accompanied by a student action 
code (SAC) . The SAC is a three digit code that indicates the type 
of action taken by the board and in those cases when a student is 
dropped from training, the SAC will also indicate why the student 
was dropped. The student action codes were expanded on 1 October 
1990 by CNET to give a more accurate picture of school attrition 
patterns. The SACS are separated into two categories, academic and 
nonacademic, and the nonacademic category is further separated into 
sub-categories for motivation, medical, administrative, and 
disciplinary reasons. When a school drops a student from training, 
the ARB may give an academic SAC or a nonacademic SAC from the 
motivation category. A nonacademic SAC assigned by an ARB should 
not be confused with a nonacademic SAC for disciplinary reasons, 
which would be assigned by an MRB. Following the approval 
required for the various board recommendations, the SAC is entered 
into the Navy Integrated Training Administrative System (NITRAS) 
database, which is 
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updated daily. NITRAS is the data base for all Navy training, that 
provides Naval training administrators a means to track the 
progress of students' training through their Naval careers. The 
data can be aggregated by school to determine attrition and setback 
patterns for every course offered. 

The accuracy of the attrition information that is input into 
the database is directly related to the care of the ARB 
procedures and the proper assignment of SACs by the individual 
training commands. The concern over the accuracy of the NITRAS 
database is reflected by the following extract from a CNTT 
instruction : 

The accuracy and care involved in student coding at the ARB 
level is fundamental to the creation of an accurate and 
meaningful data base of attrition information. This 
information may be used as a basis for administrative and 
management decisions and research studies, and it may 
influence assignment procedures. [Ref. 9:p. 2] 

This research will address the accuracy of these data in the 

context of decision making at ARBs. 

3 . Decision Making 

Naval training schools rely on a small groups to make 
decisions concerning attrition. Because of this, individual 
differences and group dynamics become a part of the decision making 
process. Ideally, a board's decision is made free of any 

influences other than those brought to the board, i.e., the 
student's academic record and the results of the board's interview 
with the student. However, there may be other 
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influences, some from outside the board, and some within the board, 
that affect the board's ultimate recommendation. For example, 
serving as the chairman may give that individual some influence 
over the other board members. By instruction, the chairman is the 
senior member present and may be an immediate superior to one of 
the other board members. Similarly, one board member may influence 
other board members through strength of conviction. Or, one or 
more of the board members 

may empathize with the student's plight due to having similar 
demographic characteristics or a similar background. [Ref. 10] 
Individuals with approval/disapproval authority may influence the 
board's decision through actions on past board recommendations, or 
through other forms of communication of their wishes. Perceptions 
of the school's effectiveness by external and internal 
organizations may drive a board's decision. 

Thus, groups dynamics, which are known to impact decision 
making, would most certainly be expected to be present in the 
process of making a decision at an ARB where a large component of 
subjective evaluation is required. Poor communications, status of 
group members, conformity, group polarization, group experience, 
and individual personalities could all impact the standardization 
of ARB decisions. [Ref. 10] 

B. RESEARCH QUESTIONS 

The present research was initiated because of CNTT ' s 
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concern over the accuracy and standardization of attrition 
reporting particularly in the "A" schools. In response to CNTT's 
concerns, three questions are examined: 

1. Do differences exist among the schools' ARB procedures 
that could promote either inaccuracy or decreased standardization 
in attrition reporting? 

2 . How much agreement is there among the ARB members 

concerning the evaluation of student characteristics for a 

given ARB decision? 

3. What other information can be captured that sheds light 
on the attrition ARB decision? 

By determining how school personnel make their decisions during the 
ARB process, information can be acquired that will help improve the 
attrition reporting process. 
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II. METHODOLOGY 



A. SUBJECTS 

The subjects of this study were the ARB members from eight 
"A" Schools and the Advanced Electronics School (AES) , which 
incorporates many of the electronic "C" Schools. The "C" 
schools were included to determine if they conduct their ARBs 
differently than the "A" schools. Seven of the "A" Schools and 
AES are part of Seirvice School Command (SERVSCHLCOM) , in San 
Diego, California. The other "A" School is Data Systems "A" 
(DS-A) school, which is part of Combat Systems Technical 
School Command located at Mare Island, California. 
SERVSCLCOM, is the largest training command that provides 
initial skill training on the West Coast. It is home for the 
following "A" Schools: Radioman "A" (RM-A) , Interior 
Communications "A" (IC-A), Data Processor "A" (DP-A) , Mess 
Specialist "A" (MS-A) , Molder "A" (ML-A) , Pattern Maker "A" 
(PM-A) , and Machinery Repairman "A" (MR-A) . The output from 
the seven "A" Schools represents 20 percent of the FY89 
graduation total for all CNTT controlled "A" Schools. [Ref. 
11] RM-A and IC-A are among 15 "A" schools that have the 
highest attrition rates. 

The study surveyed 91 ARB members, who represented varying 
percentages of their school's instructional staff. The 
representation ranged from 5.3 percent from the AES to 71.0 
percent for IC-A school. Appendix A provides a complete list 
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of the school's instructional staff representation. However, 
not all the instructional staff may be qualified to sit on an 
ARB. Discussion with the various school representatives 
indicated that the study captured most of the eligible ARB 
members associated with the respective schools. 

The ARB members averaged 14 years of service with 25 
months at their present command. Forty seven percent of the 
respondents were Second Class and First Class Petty Officers, 
32 percent were Chief Petty Officers, 17 percent were Senior 
Chief and Master Chief Petty Officers, and four percent were 
Warrant or Line Officers. Additionally, their ARB experience 
averaged over 75 ARBs and those who had experience as the 
chairman had over 40 ARBs in that capacity. 

The ARB members were separated into four groups for the 
analysis (see Table 1) because several schools (ML-A, MR-A, 
PM-A, DP-A) are small and have few ARB members. The schools 
were grouped by those that report to the same Officer in 
Charge (OIC) . At SERVSCHLCOM, ML-A, MS-A, MR-A, AND PM-A are 
known as the 3300 level schools and report to the same OIC; 
IC-A and DP-A also report to the same OIC. The information 
gathered from DS-A school was evaluated only with respect to 
ARB procedures. 
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TABLE 1 



SCHOOL GROUPINGS 

Group 1 
Group 2 
Group 3 
Group 4 

B. QUESTIONNAIRE 
The ARB members responded to a 29 item questionnaire, 

which is presented in Appendix B. The design of the 
questionnaire was based on information gathered from two sets 
of interviews with the ARB members from DS-A School. The 
first set of interviews was conducted to gather information 
about ARB procedures. The second set of interviews included 
observation of several ARBs and follow-up questions between 
each board to further define the student characteristics being 
considered when evaluating each student. Uncertainties in the 
wording of some questions on the questionnaire were examined 
with assistance from CNTT. 

C. PROCEDURES 

The questionnaires were administered to ARB members at the 
four groups of schools on separate days. The respondents 
received an initial briefing on the purpose of the study, with 
specific emphasis on the study objectives. Subjects were 
encouraged to try to add relevant comments that had not been 



RM-A 

IC-A, DP-A 

(3300 Level) MR-A, MS-A, PM-A, ML-A 
AES 
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asked on the questionnaire. No time limit was set on 
completing the questionnaire, however, all respondents had to 
complete it before leaving the room. 

The first four questions required the respondents to 
evaluate nine student characteristics shown in Table 2, 
specifically focusing on the value each factor contributes to 
the decision making process. The values for the responses 
ranged from 1 = not at all important — rarely used, to 
5 = extremely important — critical factor. The nine factors 
were evaluated over four possible types of decisions: (1) to 
drop a student from training for academic reasons, (2) to drop 
a student from training for non-academic reasons, (3) to 
setback a student for academic reasons, and (4) to setback a 
student for non-academic reasons. 

The remaining 25 questions asked the respondents to 
elaborate on how they judge certain student characteristics, 
and how their organization's ARB and attrition reporting 
systems work. 



15 



TABLE 2 



STUDENT CHARACTERISTICS CONSIDERED BY ARB MEMBERS 



1. 


Academic record 


acadrec 


2. 


Military record 


milrec 


3. 


Personal information about the student 




persinfo 




4 . 


ASVAB scores 


asvab 


5. 


Amount of night study 


nghtstdy 


6. 


Recommendations made to the board 


rectobrd 


7 . 


Professional judgement about whether 
the student will make a good sailor 


prf jdgment 


8 . 


Student attitude/motivation 


stdtmot 


9. 


High school graduate (or not) 


hsdg 
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III. RESULTS AND DISCUSSION 



The results of this study are presented in the context 
of the three study questions proposed in the introduction: 

1. Do differences exist among the schools' ARB procedures 

that could promote either inaccuracy or decreased 
standardization in attrition reporting? 

2. How much agreement is there among the ARB members 

concerning the evaluation of student factors for a 
given ARB decision? 

3. What other information can be captured that sheds light 

on the attrition process? 

A. PROCEDURAL DIFFERENCES 

A number of distinct differences were found among the 
schools' ARB procedures, which could potentially affect the 
standardization of attrition reporting. The information 
addressed in this section was gathered in part from general 
discussions with school personnel. Additionally, some of the 
data generated by the questionnaire that was administered (see 
Appendix B) are presented here. Differences among specific 
schools are included where they are relevant to the 
discussion . 

First, who is present at a student's ARB may affect the 
board's decision. In addition to the required board 
composition, some schools have the student's instructor 
present to elaborate on his/her recommendations concerning 
the student's academic 9 abilities and to answer other 
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specific questions that may shed more light on the student's 
problems . 

Just as important in influencing a board's decisions are 
those incidents that limit the board's make-up, i.e., assign- 
ing one to two individuals as permanent chairperson for all 
ARBs, which was the policy at one school sampled. The 
chairman of an ARB has positional power over the other board 
members and in that capacity might exert more influence over 
the board's decision. Therefore, the ARB's decisions over 
time could reflect individual biases that those members would 
consistently bring to each ARB. 

The manner of conducting an ARB may also affect a 
board's decision. Some of the schools conduct their ARBs in 
a relaxed, congenial atmosphere. This is done in an attempt 
to promote a more open discussion with the student, which 
would allow the board to accurately assess the student's 
problems and future potential. Other schools conduct the 
board in a more traditionally military environment with the 
student standing in front of the board members at parade rest. 
The intent of this approach is to create a professional 
environment. Some schools use a checklist to evaluate their 
ARB's conduct and procedures. This evaluation by the school's 
instructional staff occurs several times each month. 

The length of an ARB can vary from board to board, the 
shortest board may last only a couple of minutes, while some 
boards last as long as one and a half hours. Table 3 shows 
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the average length of time an ARB usually lasts for a given 
student. On average, 70 percent of the boards last between 10 
and 30 minutes, 26 percent last longer than 30 minutes, 
while approximately three percent of the boards last less than 
ten minutes. There were no time differences as a function of 
the school at which the board was held. 

TABLE 3 

AVERAGE LENGTH OF AN ARB 
(Question 13) 

During an ARB, about hov; much time is spent on each student? 
Give a range (max/min) and an average. 



less than 10 minutes 3.5% 
10 to 20 minutes 36.8% 
20 to 30 minutes 33.3% 
30 to 40 minutes 13.8% 
greater than 40 minutes 12.6% 



Percentages are based on 87 responses. 

As noted earlier, the student's time is divided between 
academic classes and duties in the ASMT. However, not all 
schools examine the student's military performance to the same 
extent at an ARB. Some schools have ASMT instructional staff 
sit on the ARBs as voting members. These schools believe they 
receive more information and can make a more accurate 
determination about the student's training status. 
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Another indication of a school's evaluation of the student's 
military performance is the amount of communication between 
the academic sections and the ASMTs . From Table 4, it can be 
seen that RM-A school and IC/DP-A school have the least amount 
of communication with their associated ASMTs. Conversely, the 
3300 level schools (MR-A, MS-A, PM-A, and ML-A) have 
considerable communication with their ASMT. The AES reported 
the most communication with their associated military side, 
however, the "C" schools do not have ASMTs. They may be 
responding with respect to their accessibility to the 
student's service record. 



TABLE 4 

COMMUNICATION BETWEEN ACADEMIC SCHOOLS 
AND ASMTs (Question 25) 

How much communication is there here between the Military 
Training Divisions and the academic sections on student 
progress? 





RM-A 


IC/DP-A 


3300 


AES 


None 


61 . 5% 


14 . 6% 


6.2% 


11.1% 


Occasional 


38 . 5% 


56.2% 


25.0% 


0% 


Considerable 


0% 


29 . 2% 


68 .8% 


88 . 9% 


N 


13 


41 


16 


9 


One of the 


important 


decisions 


a board must 


face is 



whether to setback a student or drop that student from 
training. Usually, before a student is dropped from training 
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he/she will be setback at least once. The number of setbacks 
a student receives will vary depending on the individual case. 
However, the likelihood of setting back a student may also 
vary by the school. Table 5 presents the number of setbacks 
a board member would give to a student before deciding to drop 
that student from training. 



TABLE 5 

MAXII-IUM NUMBER OF SETBACKS 
A BOARD MEMBER WOULD GIVE (Question 27) 



What is the maximum 
student? 


number 


of setbacks 


you would 


give any 




RM-A 


IC/DP-A 


3300 


AES 


No setbacks 


8 . 3% 


2 . 1% 


11.8% 


9.1% 


One setback 


8 . 3% 


0 


35.3% 


45.4% 


Two setbacks 


58 . 3% 


38 . 3% 


29.4% 


36.4% 


Three setbacks 


25 . 0% 


57 . 5% 


23 . 5% 


9 . 1% 


More than three 


0 


2 . 1% 


0 


0 


N 


12 


47 


17 


11 



The table shows that the majority of the RM-A school members 
prefer to give the student two setbacks. The IC/DP-A school 
favors setting the student back three times before dropping 
that student from training. The 3300-level schools and the 
AES are less certain, with both schools slightly favoring just 
one setback before dropping the student. In some cases, when 
it seems obvious that the board will decide to setback the 
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student, the student is physically setback to the next class 
prior to the board convening. 

The last item a board must consider is the assignment of 
the SAC. The accuracy of assigning the SAC is essential for 
maintaining an accurate NITRAS database. However, Table 6 
shows a wide range of knowledge about student action codes 
across the schools. Many of the ARB members who found the 
SACs hard to use had problems with only a small number of the 
many SACs provided. 



TABLE 6 

ARB MEMBER'S KNOWLEDGE 
OF STUDENT ACTION CODES (Question 19) 



Which of the SACs 
and why? 


are confusing or hard 


to use in any 


way. 




RM-A 


IC/DP-A 


3300 


AES 


No problems 
with SACs 


25.0% 


52.6% 


53 .3% 


20.0% 


SACs are 
confusing 


8 . 3% 


23.7% 


26.7% 


0% 


Do not use 
or have no 
knowledge 


66.7% 


23.7% 


20.0% 


80.0% 


N 


12 


38 


15 


10 



Some of the lack of knowledge concerning the SACs may be 
due to the fact that at some ARBs the chairman is the only 
member who assigns the SAC. Additionally, some of the board 
members stated that they did not evaluate their students for 
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nonacademic reasons and therefore could not give a nonacademic 
SAC even though the nonacademic SAC may have been more 
appropriate for that particular student. Although, at one of 
those schools in which the board members reported not using 
nonacademic SACs, the chairman does, in fact, assign 
nonacademic SACs. 

B. AGREEMENT IN DECISION MAKING 

Besides procedural differences, differences in the ARB 
members' opinions about the importance of certain student 
factors for making a decision about a student may also 
contribute to either a lack of standardization or inaccurate 
attrition reporting. These factors were mentioned in Chapter 
II, e.g., the student's academic record, recommendations made 
to the board, etc. If there is little agreement among the 
board members, then it is highly likely that there is a great 
amount of variation in attrition reporting. There is a finite 
amount of information the ARB has available by which it can 
evaluate a student. Therefore, differences in weighing a 
student factor from one set of ARB members to another may 
elicit varied decisions. 

Two sets of analyses were performed on the data. The 
first analysis consisted of the Friedman Two-Way Analysis of 
Variance by Ranks. This test determined whether the ARB 
members value all the student factors equally or if they give 
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some factors more weight than others. The values for the nine 
student factors were ranked for each ARB member and then the 
mean rank for each student factor was calculated. In this 
case the most highly valued student factor could receive a 

9.0, while the least valued student factor could receive a 

1.0. From the mean ranks, a test statistic with an 
approximate Chi-square distribution was calculated with the 
following formula: 

12 ^ 

= Nk (k + 1) E (r.)2 - 3^ (k + 1) 

j=l 

where : 

N = number of ARB members, 
k = number of student factors, and 

Rj = sum of ranks in jth column. [Ref. 12:pp. 167- 

171] 

The second analysis conducted was the Kendall's W 



Coef f icient 


of 


Concordance . 


Kendall ' s W 


served as a 


measurement 


of 


agreement among 


the 


raters 


concerning the 


importance 


of 


the given student 


factors . 


A Kendall 



coefficient of W = 1.0 would indicate perfect agreement among 
the raters. Conversely, a coefficient of W = 0 would indicate 
no agreement. The procedures for calculating Kendall's W are 
similar to the Friedman test except that after the mean ranks 
are calculated, the sum of squares of the observed deviations 
from the mean ranks are calculated. From that information. 
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Kendall's W may be calculated by the following formula: 



s 

W = l/12k‘^ (N-^ - N) 



where : 



s = sum of squares of the observed deviations from 
the mean of R., 



= Z (R. - ZR.)2 
N 



k = number of ARB members. 

N = number of student factors judged. 

R. = sum of ranks, j = 1 to 9. [Ref. 12:pp. 229- 

237] 

The analyses were performed for each of the four types of 
decisions from command-wide rankings, and then repeated by 
individual school groups. 

1 . Analysis of Comma nd-k'ide Responses 

The first set of analyses considered the responses 
command-wide, i.e., all schools were combined. The results 
are shown in Table 7. The Friedman's Test indicated that the 
ARB members do not value all of the student factors equally. 
In other words, there are significant differences in the level 
of importance assigned to student factors for each of the four 
types of decisions. The data in Table 7 show the relative 
ranks followed by the mean rank of each student factor for 
each of the four types of decisions (1-9 with 1 as the most 
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important).’ The presentation of the data will begin with a 
discussion of the rankings of student factors shown in Table 
7, followed by a more detailed discussion of specific student 
factors, and finally a discussion of the levels of agreement 
among the ARB members as indicated by the Kendall's 
coefficient. All statistical values shown on Table 7 are 
significant at the level of p < .01. 

TABLE 7 

RELATIVE AND MEAN RANKINGS 
OF STUDENT FACTORS FOR TYPE OF DECISION 



Student 


Academic 

setback 


Nonacademic 

setback 


Academic 

drop 


Nonacademic 

drop 


acadrec 


2/2.77 


6/5.17 


1/2 . 24 


6/5.42 


milrec 


7/5.68 


2/3 . 59 


7/5.80 


2/2.44 


persinfo 


4/4 . 89 


3/3 . 87 


5/5.04 


3/3.73 


asvab 


8/6.23 


8/7.40 


8/6.44 


8/7.41 


nghtstdy 


3/4.70 


7/6.05 


3/4 . 54 


7/6.38 


rectobrd 


5/5 . 10 


4/4 . 09 


4/4.88 


5/4 . 61 


prf jdgmnt 


6/4 . 26 


5/4 .51 


6/5.36 


4/4.49 


stdtmot 


1/2 .29 


1/2.39 


2/2 . 54 


1/2.27 


hsdg 


9/7 . 97 


9/7.93 


9/8 . 16 


9/8 .26 


N 


89 


76 


88 


75 


df 


8 


8 


8 


8 


Friedman ' s 


Chi square 


270 . 04 


269 . 84 


209.06 


345.28 


Kendall ' s 


Coefficient 


.42 


. 50 


. 50 


. 64 


Chi square 


302 . 22 


301.44 


354.88 


385.36 



’The mean rankings produced by the Friedman's analyses 
were subtracted from 10.0 to facilitate comparisons with the 
relative rankings. 
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For the academic type decisions, the student's academic 
record and attitude/motivation received similar mean ranks and 
were ranked much higher than any other student factor 
considered. In other words, academic record and motivation 
were ranked as the most important factors to consider when 
making a decision to setback or to drop a student for academic 
reasons . 

For the nonacademic type decisions, the student's 
attitude/motivation was the most important factor for both 
nonacademic decisions. The student's military record was 
ranked very closely to student motivation for the nonacademic 
drop from training decision, and to a lesser extent for the 
nonacademic setback decision. Also, personal information was 
valued more strongly than the other student factors for the 
nonacademic type decisions. 

Several student factors were ranked very closely together 
and comprised a mid-ranged group for a given type decision. 
The mid-ranged group included the remaining student factors, 
with the exception of those noted above, and the two lowest 
ranked student factors. The student's ASVAB scores and 
whether or not the student was a high school graduate (HSDG) , 
were consistently ranked as the least important student 
factors to consider when making ARB decisions. 

As noted, the student's motivation was ranked the highest 
for three of the four possible decisions and it was ranked 
second for the fourth decision. Even though there is 
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agreement about the value of evaluating student motivation (as 
will be shown below in the discussion of Kendall's 
coefficient) , it is a highly subjective judgment that is rated 
by a number of different factors by the ARB members. There 
were over 25 different responses on what constitutes 
attitude/motivation. Table 8 presents the most common 
responses, by percentage of the total number of 210 responses 
received. Many of the ARB members gave more than one 
response . 



TABLE 8 

HOW MOTIVATION IS JUDGED 
(Question 9) 

How do you judge motivation (besides night study) . 



Participation in class 25% 
Seeking help from instructors 19% 
Completing homework 17% 
Demonstrates extra effort 15% 
Comments from instructors 10% 
Rewrites class notes 7% 
Miscellaneous 7% 



Class participation was the most frequently 
mentioned measure of student attitude. It is followed by the 
group of measures shown on the table. Other, less common 
responses included whether or not the student makes eye 
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contact at the ARB, how sharp the student looks in uniform, 
and the "gut feeling" of the ARB member. 

The student's ASVAB scores were ranked very low in 
importance for any type of decision being made. This seems 
surprising because the ASVAB scores are used as an entrance 
screen for most schools. Table 9 presents the responses 
concerning the usefulness of ASVAB scores for determining if 
the student's academic problems are legitimate or really a 
lack of motivation. The data are based on percentages of the 
91 people who responded to the question. Over half of the 
respondents thought a student's ASVAB scores would be useful 
in judging student motivation. However, this contradicts the 
low mean rankings given to it by the same individuals. 

TABLE 9 

VALUE OF THE STUDENT'S 
ASVAB SCORES TO THE ARB (Question 6) 

Do ASVAB scores help you deterinine whether a student's 
academic problems are real or due to lack of motivation? 



ASVAB scores 


are helpful 


52 . 6 


ASVAB scores 


are not helpful 


36.8 


Not sure 




10.6 



It would appear that, conceptually, ARB members see 
value in ASVAB scores as an aid in decision making, which is 
reflected in Table 9. However, when compared to other sources 
of information on a relative basis, ASVAB scores are ranked at 
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a very low level (see Table 7) . 

Military record was rated highly for its value in 
making nonacademic type decisions. Yet, the manner in which 
information is obtained about the military record was varied. 
As stated earlier, some schools have one of the ASMT's 
instructional staff sit on the ARBs as a voting member. Other 
schools call the ASMT whenever the student is having academic 
problems to discover if the student is also having other 
problems that may have a contributing influence to the 
student's academic problems. The ARBs may also have the 
student's military record present at the board. The student's 
military record would contain information about the student's 
military performance, specifically noting any infractions the 
student might have committed. In some of the smaller schools, 
the academic instructors usually hear when a student is having 
problems at the ASMT. 

When a student comes before an ARB, one of the first 
objectives is to determine the kinds of personal problems the 
student is having, if any. A student's personal problems may 
include family, financial, and medical problems. Table 10 
presents the kinds of personal information that could help an 
ARB make a decision whether to setback or drop a student. 
Since ARB members could give more than one answer to this 
question, percentages are based on 245 responses. 
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TABLE 10 



PERSONAL INFORMATION USED 
BY THE ARB (Question 5) 

What kind of personal information about the student might 



help you make a decision during an ARB? 

A student's personal problems 56% 

Attitude in class, 

study habits, effort 14% 

Background (family, jobs, hometown) 7% 

Education background 4% 

Misc. (substance abuse, friends, 
after hours' habits, stress, 

depression, and goals) 19% 



How a student is judged as a future sailor may have 
an affect on the ARB's decision. This factor was ranked 
between fourth and sixth as shown on Table 7. Table 11 
presents the types of factors considered in making judgments 
about the student as a future sailor. The ARB members 
provided over 45 different factors used to make this judgment. 
The most common responses are shown on Table 11. The 
percentages shown on the table are based on 250 responses. 

As described earlier, these data represent an 
average of all survey respondents. Thus, the next step in 
analyzing the data was to investigate the extent to which 
people agree on this set of averaged rankings. A simple 
inspection of the data indicates that there is not perfect 
agreement. For example, while motivation was ranked very 
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highly for three out of the four decisions, not everyone 
rated it as first (the values shown on Table 7 range from 2.27 
to 2.39). However, it is not clear how much disparity there 
is . 



TABLE 11 

FACTORS CONSIDERED WHEN MAKING 
PROFESSIONAL JUDGMENTS (Question 8) 

How do you judge whether someone will make a good sailor? 

Shows a positive attitude to complete 



the training 33% 

Motivation, willingness to work, 

desire to excel, initiative 19% 

Appearance and military bearing 17% 

Behavior, class performance, 
study habits, follows direction, 

night study, asks questions 13% 

Military record 7% 

Various personality traits 6% 

Respectful to seniors 5% 



The Kendall's W provides a quantification on the 
level of agreement. The Kendall's coefficients ranged from a 
moderate level of agreement for the academic setback decision 
W = .42, to a higher level of agreement for the nonacademic 
drop from training decision W = .64. In other words, the 
Kendall's Statistic applied to these data indicates that there 
is a level of agreement that is significant, or non-random, 
for all decisions. Further, there is a higher level of 
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agreement among decision makers when dropping a student for 
nonacademic reasons as compared to other decisions. 

2 . Analysis of School-Level Responses 

The remaining discussion of the level of agreement 
will focus on the four groups of schools. Particular 
attention will be paid to any differences that exist between 
the schools' rankings of the student factors. The Friedman's 
Two-Way Analysis of Variance by Ranks and Kendall's W 
Coefficient of Concordance were significant for all schools at 
the one percent level unless noted. 

a. Setback for Academic Reasons 

The schools' patterns of ranking the student 
factors (see Appendix C) followed the command-wide pattern 
with a few exceptions. For all schools, the highest ranked 
student factors were the student's academic record and the 
student's attitude/motivation. The mid-ranged student factors 
were also similarly ranked v/ith the exception of RM-A school. 
RM-A school gave the student's ASVAB scores slightly higher 
mean and relative rankings. The other schools ranked the 
ASVAB scores as lowest in importance. Also, RM-A school 
valued the student's military record and their own 
professional judgment less than the other school groups. 

The levels of agreement within the schools were 
statistically significant indicating that there is a non- 
random basis for ranking the student factors among the 
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school's ARB members. The least amount of agreement occurred 
within the 3300-level schools (W = .43), while the AES had the 
highest level of agreement (W = .54). 

b. Setback for Nonacademic Reasons 

The data table discussed in this section are 
presented in Appendix D. The RM-A school data were not 
statistically significant, most probably due to the small 
number of respondents, therefore the results from RM-A school 
will be omitted from the discussion. 

The schools' patterns of ranking the student 
factors were similar to the command-wide rankings, with the 
exception of the AES. Student attitude/motivation was the 
highest ranked student factor for all the schools, and the 
student's military record was the second highest ranked 
student factor. The AES agreed with the ranking of the 
student's attitude/motivation as the highest student factor, 
but differed on the next two most important student factors. 
They valued their own professional judgment about the student 
and, to a slightly lesser extent, recommendations made to the 
board, much more than the other school groups. The remainder 
of the mid-ranged student factors were patterned similarly to 
the command-wide rankings. Also, all the schools ranked the 
student's ASVAB scores and HSDG as the least important of the 
student factors. 

For this type of decision, the levels of 
agreement within each school were similar to each other and to 
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the command-wide rating with the exception of the AES, which 
had a high level of agreement (W = .73). 

c. Drop for Academic Reasons 

Once again the schools' patterns of ranking the 
student factors were very similar to the command-wide rankings 
(see Appendix E) . The student's academic record and the 
student's attitude/motivation were ranked closely as the most 
important student factors. The mid-ranged student factors 
were ranked similarly to the group as a whole, and ASVAB 
scores and HSDG ranked as the least important student factors 
among all the schools. 

The levels of agreement across the schools 
varied from RM-A school with the least level of agreement 
(W = .47) to the highest level at the AES (W = .59). There is 
slightly more certainty about the ranking of the student 
factors for the academic drop from training decision than 
there was for the academic setback decision. 

d. Drop for Nonacademic Reasons 

The data table summarized in this section is 
presented in Appendix F. The RM-A school data consisted of 
only two responses, therefore these results will not be 
discussed . 

The schools' patterns of ranking the student 
factors closely matched the command-wide pattern of student 
factor rankings. The student's military record and the 
student's attitude/motivation were ranked closely and were the 
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two most important student factors. The schools ranked the 
mid-ranged student factors similarly, and ASVAB scores and 
HSDG were the least important student factors. 

This type of decision produced the highest 
levels of agreement among the schools' ARB members. The 3300- 
level schools had the lowest level of agreement (W = .62), 
while the AES had the most agreement (W = .77). 

Table 12 summarizes the Kendall's W Coefficient 
of Concordance statistics for the command-wide analysis and 
then for each school group. Overall, the analyses of the data 
from each separate school show the following trends: 

1. RM-A school data were omitted from the discussions of 

nonacademic decisions due to the small number of 
respondents. RM-A instructional personnel declined to 
respond to these items because, at their ARBs, they do 
not evaluate the students for nonacademic reasons. 

2. Agreement was only slightly higher within schools as 

compared to results produced by the command-wide 
analysis, with the exception of the AES. The AES had 
consistently higher agreement across all types of 
decisions . 

3. There is more agreement among the ARB members concerning 

the importance of student factors for the decisions to 
drop a student from training as compared to setback 
decisions . 

4 . There is more agreement among ARB members concerning the 

importance of student factors for nonacademic as 
compared to academic decisions. 
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TABLE 12 



AGREEMENT COEFFICIENTS BY SCHOOL GROUP 
AND TYPE OF DECISION 



Command- 
Decision Wide 


RM-A 


IC/DP-A 


3300- 

level 


AES 


Academic 


Setback .42 


.45 


.46 


.43 


. 54 


Nonacademic 
Setback .50 


.47 


. 50 


.48 


.73 


Academic 


Drop .50 


.47 


. 52 


. 55 


. 59 


Nonacademic 
Drop . 64 


.79 


. 65 


. 62 


.77 



C. ADDITIONAL ATTRITION-RELATED QUESTIONS 

There is a grov;ing concern by people involved with Navy 
enlisted training that, due to the addition of a SAC for 
voluntary disenrollment , there will be an increase in the 
number of students desiring to disenroll from training. A 
realistic job previev; (RLJP) is one instrument that has been 
used to prevent that situation from occurring by portraying 
the perspective workplace through lectures, books, videos, 
etc. The ARB members were asked what effect they thought a 
RLJP would have in preventing attrition at their school. 
Their responses are given in Table 13. At least 60 percent of 
the ARB members thought a RLJP could be useful or would be 
very useful in preventing attrition at their school. The RLJP 
received the most support from the 3300-level schools and the 
least support from the AES (obviously, a RLJP would have less 
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value in a "C" School). 



TABLE 13 

VALUE OF A RLJP (Question 29) 

How useful would a realistic job preview for this rating be 
in preventing attrition for any reason? 





RM-A 


IC/DP-A 


3300 


AES 


Not useful 


15.4% 


17 . 0% 


11.8% 


40.0% 


Could be useful 


46.2% 


23 .4% 


11.8% 


20.0% 


Very useful 


38 .4% 


59 . 6% 


76.4% 


40.0% 


N 


13 


46 


17 


10 



How the instructors and ARB members feel about the 
difficulty of their curriculum may affect their opinions about 
the students. Table 14 presents the percentages of ARB 
members who feel that the curriculum at their school is either 
too hard, too easy, or about right. 
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TABLE 14 



DIFFICULTY OF THE 
SCHOOL'S CURRICULUM (Question 28) 

In consideration of what your students will have to do when 
they eventually perforin in their rating, rate the curriculum 
here . 







RM-A 


IC/DP-A 


3300 


AES 


a . 


too hard 


8 . 3% 


31.3% 


0 


9.1% 


b. 


too easy 


75.0% 


8.3% 


0 


81.8% 


c . 


about right 


16.7% 


60.4% 


100% 


9.1% 


N 




12 


48 


17 


11 



The final question asked the ARB members whether their 
school's attrition had gone up, down, or stayed about the 
same, and why. The responses are presented in Table 15. 



TABLE 15 

ARB MEMBER'S PERCEPTIONS 
ABOUT THE CHANGE IN ATTRITION (Question 24) 

In the time that you have been here, has attrition gone up, 
down, or stayed about the same? Why? 





RM-A 


IC/DP-A 


3300 


AES 


Stayed the same 


0% 


9.8% 


29.4% 


100 


Down, due to: pressure 
or lowered standards 


83 . 3% 


51.2% 


0% 


0 


Down, due to: improved 
students or methods 


16.7% 


9.8% 


64 .7% 


0 


Gone up 


0% 


29 . 2% 


5.9% 


0 


N 


12 


41 


17 


11 
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About 83 percent of the respondents from RM-A school and close 
to 51 percent of the respondents from IC/DP-A school indicated 
that attrition had gone down. The two reasons given for the 
decline in attrition were lowered grading standards and the 
pressure they had received from their superiors to lower 
attrition. Twenty nine percent of the respondents from IC/DP- 
A school stated that attrition had increased. Conversely, 
over 66 percent of the 3300-level schools respondents 
indicated that attrition had gone down due to improved 
training methods and extra effort from the instructors. All 
of the AES respondents stated that their attrition has 
remained about the same. 

It is interesting to note that the RM-A and IC-A schools 
are among the 15 schools that have high attrition, and also 
have members who are the most concerned about pressure to 
reduce attrition through reduced lowered standards. These 
instructors have apparently made some assumptions concerning 
attrition policy. 

While the data collected for the last three questions 
presented here are only indirectly related to attrition 
reporting, they are nonetheless highly relevant to the 
decisions made at ARBs. For example, an instructor who 
perceives that standards have been lowered to reduce attrition 
may be influenced in two ways. First, if the instructor 
believes that standards have already been lowered for course 
tests (or curriculum) , he/she may be more likely to be 
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unnecessarily stringent in standards applied to the decision 
to setback or drop. The result could be inappropriate 
attrition . 

On the other hand, if the instructor perceives 
incorrectly that the concern with reducing attrition is such 
that it is his/her job to apply lowered standards, that person 
may play a part in creating a problem that doesn't exist. The 
result in this situation would be to reduce attrition at the 
expense of quality student output--the very outcome of concern 
to a number of the instructors surveyed. 

Thus, indirect factors can affect attrition rates in 
non-optimal ways. These issues would seem to merit additional 
exploration . 
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IV. 



CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

This research examined the school's attrition process, 
specifically focusing on those areas that could result in either 
inaccurate attrition reporting or decreased standardization among 
the attrition reporting schools. The study looked at three general 
areas at one Navy training organization: 1) procedural 
differences, 2) differences in the levels of agreement concerning 
various student factors, and 3) ARB member's perceptions. 

1 . Procedural Differences 

The ARB procedures differ across schools. Those differences 
may contribute to decreased standardization in attrition reporting 
among the schools. Most of the differences appear to be due to 
varying interpretations of CNTECHTRA INST. 1540. 46A, while others 
exist because of the school's policy, e.g., the chairman as the 
only member to assign the SACs, or not assigning a nonacademic SAC 
because it is an Academic Review Board. 

2 . Levels Of Agreement 

Without an explicit policy governing what student information 
should be considered, there exist remarkable similarities among the 
schools concerning the value certain student factors contribute to 
the ARB's decision making process. However, the ARB member's 
judgement about the characteristics that make-up some of those 
student factors varies. Also, the ARB members within the schools 
have moderate to high levels of agreement concerning the importance 
of the student factors. 
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3. 



ARB Member's Perceptions 



The board members' perceptions about the school's 
effectiveness in performing its mission may affect the accuracy of 
attrition reporting. Specifically, perceptions about course 
difficulty and lowering standards to meet attrition goals, may 
influence instructor morale and ARB decision making. 

4 . Outcomes Of ARB Decisions 

Finally, it is important to add that the accuracy of current 
ARB decisions is unknown. Even if standardization of decisions 
within and across ARBs could be achieved, there have been no 
measures of the outcomes of any decisions made, i.e., there is no 
way of knowing if decisions made are correct. 

B. RECOMMENDATIONS 

1. Consider the differences in board procedures to determine 
if there may be value in changing procedures to be 
consistent across all schools. The following areas 
should be reviewed: 

a. Should a student's instructor be present at the 
board? 

b. Should permanent chairpersons be assigned for all 
ARBs, or should the job be rotated? 

c. Is it more useful to conduct a board in a relaxed, 
informal manner or in a more traditionally 
military, formal environment? 

d. To what extent should an ARB consider military 
performance, and how should the board acquire that 
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information? 



e. Should policy be established with respect to number 
of setbacks allowed for each student? 

f. Should all board members be equally knowledgeable 
about SACs? 

g. Should the chairman, vice the board, have sole 
responsibility for assigning the SAC? 

2. Consider the value placed on each of the student factors 
in decision making to determine if the rankings are consistent with 
the best interests of Navy training. 

3. Given the consistently high level of agreement between 
AES board members, determine if there is something about the way 
they conduct their ARBs that can be transitioned for use at the "A" 
schools . 

4. Explore expanded use of realistic job previews as a tool 
for reducing attrition. 

5. Explore ARB members' understanding of attrition policy 
impl ications . 

6. Investigate the fleet performance of marginal students, 
as determined by test scores or number of setbacks. 

7. Conduct a cost analysis. The cost effectiveness of 
setting back students from training, as opposed to dropping them, 
might provide useful information for decision making. 

8. Determine the generalizability of the present data to 
other enlisted training schools. 

This study is the initial attempt at analyzing the intricacies 
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of decision making at ARBs. The follow-on studies recommended 
would take the issues and questions addressed here to the next 
logical level of analysis. Specifically, performance and cost data 
associated with students who have academic difficulties would 
provide a concrete basis from which to derive policy concerning 
acceptable levels of attrition at Navy "A" schools. 



45 



LIST OF REFERENCES 



1. Center for Naval Analyses, CRM 86-26, An Evaluation of 
the Effectiveness of Classroom and On-The-Job Training. 
by Q. Aline, and A. J. Marcus, February 1986. 

2. Office of the Assistant Secretary of Defense . Military 

Manpower Training Report 1991 . 1990. 

3. Center for Naval Analyses, CRM 88-76, The Role of Student 

Quality in A-School Training Attrition: Trends in 

Selected Ratings . Byrnes, P.E., Marcus, A.J, and Thomason, 
J . E . , July 1988 . 

4. Training Analysis and Evaluation Group, Vol. 75, The 
Relationship of Personality Characteristics to Attrition 
and Performance Problems of Navy and Air Force Recruits . 
Spielberger, C.D., and Barker, L.R., September 1979. 

5. Center for Naval Analyses, CRM 88-149, A-School Attrition 
and Results . Byrnes, P.E., and Marcus, A. J. , September 
1988 . 

6. CNET MSG 2521582, June 1987. 

7. Navy Personnel Research and Development Center, in press. 

The Model School Process , by Van Matre, N., 1990. 

8. CNET MSG 1720502, July 1989. 

9. CNTECHTRAINST 1540.46A. 

10. Wright, G. Behavioral Decision Making, Plenum Press, New 
York, 1985. 

11. Chief of Naval Technical Training, NITRAS Extract . 

12. Siegel, Sidney, Nonparametric Statistics for the 
Behavioral Sciences . McGraw-Hill Book Company, 1956. 



46 



APPENDIX A 



INSTRUCTIONAL STAFF REPRESENTATION BY SCHOOL 



SCHOOL 


TOTAL 

STAFF 


TOTAL 

STUDIED 


PERCENT : 


RM-A 


102 


14 


13.7 


IC-A 


62 


44 


71.0 


DP-A 


8 


4 


50.0 


MS-A 


33 


6 


18.2 


PM/ML-A 


10 


3 


30.0 


MR-A 


20 


9 


45.0 


AES 


209 


11 


5.3 
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APPENDIX B 



ARB QUESTIONNAIRE 



The purpose of this survey is to look at how your 
organization conducts its Academic Review Boards. We are 
interested in understanding how decisions are made. In other 
words, what kind of information do you use in making a 
decision? We are particularly interested in the importance 
you give to different factors and any unique factors you may 
consider. This will allow us to better understand the 
decision making process. 

Your answers are anonymous, your command will not have 
any access to these questionnaires, and any information 
reported will be aggregated so no one answer will be singled 
out. 

GENERAL INFOR^!ATION 

Rank/Paygrade 

Years of service 



Time at this command 



Approximate number of times you 
have sat on an ARB 



Approximate number of times you have 
served as chair of an ARB 



THE ARB PROCESS 



Please consider the specific types of decisions described 
below and in the spaces next to each factor indicate the 
following : 

a. How important each type of information is in 
leading to that particular decision. Use a 1-5 scale where 

1 = not at all important; rarely used 

2 = somewhat important 

3 = average importance 

4 = very important 

5 = extremely important; critical factor 
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b. The rank of each of the factors compared to the 
others for that particular decision. You will have ranks 1-9 
if you use only the factors we have suggested, or more if you 
can think of factors to add on that we have neglected to 
include. We encourage you strongly to try to add factors 
anywhere you can to make sure we have an accurate 
understanding of the ARB process . 

1. For your first set of ratings and rankings, consider a 
typical situation (we know there are unique situations, try to 
focus on the average) in which the board decides that a 
student should be setback for academic reasons. Now evaluate 
how important the following factors were in helping you reach 
this decision: 

Importance Rank 



Academic record 

Military record 

Personal information 
about the student 

ASVAB scores 

Amount of night study 

Recommendations made to 
the board 

Your professional judgment 
about whether this person 
will make a good sailor 

Student attitude/motivation 

High school graduate (or not) 

Other : 
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2. For the next set of ratings and rankings, consider the 
situation in which the decision is made to setback the student 
for non-academic reasons. 

Importance Rank 



Academic record 

Military record 

Personal information 
about the student 

ASVAB scores 

Amount of night study 

Recommendations made to 
the board 

Professional judgement 
about whether this person 
will make a good sailor 

Student attitude/motivation 

High school graduate (or not) 

Other : 



3. Now consider the situation in which the decision is made 
to drop a student from training for academic reasons. 

Importance Rank 



Academic record 

Military record 

Personal information 
about the student 

ASVAB Scores 

Amount of night study 

Recommendations made to 
the board 
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Professional judgment 
about whether this person 
will make a good sailor 

Student attitude/motivation 

High school graduate (or not) 

Other 



4. Now consider the situation in which the board decides to 
drop a student from training for non-academic reasons. 

Importance Rank 



Academic record 

Military record 

Personal information 
about the student 

ASVAB scores 

Amount of night study 

Recommendations made 
to the board 

Professional judgment 
about whether this person 
will make a good sailor 

Student attitude/motivation 

High school graduate (or not) 

Other 
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17. After any disagreements are discussed and a decision has 
been made, indicate the percent of the time you feel 

a. Satisfied with the decision made: percent. 

b. That you still disagree with the decision: 

percent . 



18. Where did you learn how to participate in academic review 
boards? List all sources and check the one that provided you 
with the most/best information. 



STUDENT ACTION CODES 

19. Which of the SACs are confusing or hard to use in any 
way, and why? 



20. If you could add more SACs to those you have available to 
you (in order to increase the accuracy of the system), what 
would they be? 



21. Is there any reason you avoid using particular codes? 
Which ones, and why? 



22. Is there any reason you would lean toward using either an 
academic or non-academic drop code for a person who appeared 
to be about equally unable and unmotivated to complete the 
course? 



Which type code would you use and why? 
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SCHOOL INFORMATION 

23. What are the most common causes of attrition at this 
school? 



24. In the time that you have been here, has attrition gone 
up, down, or stayed about the same? 



If up, why? 



If down, why? 



25. How much communication is there here between the Military 
Training Divisions and the academic sections on student 
progress? 

(circle one) 

a. None 

b. Occasional 

c. Considerable 



26. From what you have seen, about what percentage of 
students with waivers are eventually dropped from training? 



About percent. 



27. What is the maximum number of setbacks you would give any 
student? 



Is that number based on your own feelings, or guidance 
from your command? 
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28. In consideration of what your students will have to do 
when they eventually go out to perform in their rating, is the 
curriculum here 

(circle your answer) 



a . 


too 


hard 


b . 


too 


easy 


c . 


about right 



29. About how often do you think the problems leading to 
attrition are a result of a student having unrealistic/ 
inaccurate expectations of what the job/rating involves? Give 
a percentage that reflects your best guess. 

percent. 



30. At this point, please add anything that we have not 
included that will help us to have a complete understanding of 
ARBs, student action codes, and the way you do business at 
this school. 



THANK YOU FOR YOUR TIME. 
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APPENDIX C 



Student 

Factor 

acadrec 

milrec 

persinf o 

asvab 

nghtstdy 

rectobrd 

prf jdgmnt 

stdtmot 

hsdg 

N 

df 

Friedman ' s 
Chi square 

Kendall ' s 
coefficient 

Chi square 



SETBACK FOR ACADEMIC REASONS 
BY SCHOOL GROUP 



RM-A 


IC/DP-A 


3300 


AES 


1/2 . 39 
8/6.82 


2/2 .34 
6/5.55 


2/3 . 89 
7/5.64 


2/3.23 

6/4.82 


3/4 . 18 


4/4 . 98 


5/4.69 


7/5.77 


6/5.43 


8/5.96 


8/6.78 


8/7 . 50 


5/4.93 


3/4 . 92 


3/4 . 28 


3/4 . 18 


4/4.39 


7/5.71 


4/4.53 


4/4.36 


7/6.04 


5/5.30 


6/4 . 92 


5/4.68 


2/3 . 07 


1/2 . 29 


1/2 . 19 


1/2 . 27 


9/7.75 


9/7.95 


9/8 . 08 


9/8.18 


14 


46 


18 


11 


8 


8 


8 


8 


44 . 24 


152 .81 


55.26 


42.18 


. 4 5 


.46 


.43 


. 54 


50.60 


169 .21 


62 .40 


47.30 
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APPENDIX D 



DROP FOR ACADEMIC REASONS 
BY SCHOOL GROUP 



Student 

Factor 


RM-A 


IC/DP-A 


3300 


AES 


acadrec 


1/2.38 


1/2 . 11 


1/1.94 


2/3.14 


milrec 


7/6.12 


7/5.51 


7/6.03 


7/6.23 


persinfo 


4/5.08 


4/5.10 


4/4.75 


6/5.23 


asvab 


8/6.35 


8/6.09 


8/7 . 00 


8/7 . 09 


nghtstdy 


5/5.31 


3/4 . 61 


3/4.17 


4/3.95 


rectobrd 


3/3.81 


5/5.34 


6/5.14 


3/3.82 


prf jdgmnt 


6/5 .65 


6/5.49 


5/5.06 


5/5.00 


stdtinot 


2/2 . 62 


2/2 .48 


2/2.83 


1/2.23 


hsdg 


9/7 . 69 


9/8 .28 


9/8.08 


9/8 . 32 


N 


13 


46 


18 


11 


df 


8 


8 


8 


8 


Friedinan ' s 
Chi square 


42.95 


168.37 


70.49 


44.87 


Kendall ' s 
coefficient 


.47 


.52 


. 55 


.59 


Chi square 


48.95 


190.96 


78.82 


51.54 
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APPENDIX E 





SETBACK 


FOR NONACADEMIC 


REASONS 




student 

Factor 


RM-A 


BY SCHOOL GROUP 
IC/DP-A 


3300 


AES 


acadrec 


5/4.75 


6/4 . 96 


7/5.35 


6/6.00 


milrec 


2/3 . 12 


2/3 .46 


2/3 .79 


4/4.05 


persinfo 


4/4 . 12 


3/3 . 68 


3/4 . 09 


5/4 .25 


asvab 


8/7 . 12 


8/7 . 33 


8/7 .38 


8/7.85 


nghtstdy 


7/6.87 


7/6 . 27 


6/4 . 97 


7/6.55 


rectobrd 


1/2 . 62 


4/4 . 16 


5/4 .71 


3/3 . 30 


prf jdgmnt 


6/5.62 


5/4 . 84 


4/4.21 


2/3.05 


stdtmot 


3/3 . 50 


1/2 .44 


1/3.35 


1/1.80 


hsdg 


9/7.25 


9/7 . 87 


9/8.15 


9/8.05 


N 


4 


45 


17 


10 


df 


8 


8 


8 


8 


Friedman ' s 
Chi square 


13.72* 


160 . 02 


58 . 28 


53.13 


Kendall ' s 
coefficient 


.47 


. 50 


.48 


.73 


Chi square 


15.20* 


179 . 13 


65.33 


58.65 



* Nonsignificant p > .05 
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APPENDIX F 





DROP FOR 


NONACADEMIC REASONS 




BY 


SCHOOL GROUP 




Student 

Factor 


RM-A 


IC/DP-A 


3300 


acadrec 


5/5.25 


6/5.15 


6/5.68 


milrec 


4/4.75 


1/2.20 


2/2.56 


persinfo 


1/2.25 


3/3.79 


3/3.88 


asvab 


8/7.75 


8/7 . 26 


8/7 . 47 


nghtstdy 


7/7.00 


7/6.38 


7/6.09 


rectobrd 


2/2 .75 


5/4.91 


5/4 . 68 


prf jdgnnt 


5/5.25 


4/4 . 66 


4/4 .26 


stdtmot 


3/3 . 25 


2/2 . 29 


1/2 . 21 


hsdg 


9/7.75 


9/8 . 35 


9/8 . 18 


N 


2 


46 


17 


df 


8 


8 


8 


Friedman ' s 
Chi square 


9.73* 


214.75 


75.92 


Kendall ' s 
coefficient 


.79 


. 65 


. 62 


Chi square 


12 . 69* 


237.66 


84 . 50 



* Nonsignificant p > .05 



AES 

6/6.25 

2/3.10 

3/3.45 

8/7.90 

7/6.75 

4/3 .44 

5/3.90 

1 / 2 . 10 

9/8 . 10 

10 

8 

54 . 24 

. 77 
61.46 
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